Computationally Efficient Feature Extraction and Matching Iris Recognition

ABSTRACT

A method and system for uniquely identifying a subject based on an iris image. After obtaining the iris image, the method produces a filtered iris image by applying filters to the iris image to enhance discriminative features of the iris image. The method analyzes an intensity value for pixels in the filtered iris image to produce an iris code that uniquely identifies the subject. The method also creates a segmented iris image by detecting an inner and outer boundary for an iris region in the iris image, and remapping pixels in the iris region, represented in a Cartesian coordinate system, to pixels in the segmented iris image, represented in a log-polar coordinate system, by employing a logarithm representation process. The method also creates a one-dimensional iris string from the iris image by unfolding the iris region by employing a spiral sampling method to obtain sample pixels in the iris region, wherein the sample pixels are the one-dimensional iris string.

CROSS-REFERENCE TO A RELATED APPLICATION

This application is a division of U.S. patent application Ser. No.13/846,837, filed Mar. 18, 2013, now U.S. Pat. No. 8,861,799, issued onOct. 14, 2014. U.S. patent application Ser. No. 13/846,837 is a divisionof U.S. patent application Ser. No. 12/426,210, filed Apr. 17, 2009, nowU.S. Pat. No. 8,411,910, issued on Apr. 2, 2013. U.S. patent applicationSer. No. 12/426,210 relates to and claims the benefit of U.S.Provisional Patent Application Ser. No. 61/073,735, titled “IrisLogarithm Representation”, and filed on Jun. 18, 2008, the disclosure ofwhich this application hereby incorporates by reference. U.S. patentapplication Ser. No. 12/426,210 also relates to and claims the benefitof U.S. Provisional Patent Application Ser. No. 61/073,734, titled“Segmentation of Iris Region for Iris Recognition Using SpiralUnwrapping”, and filed on Jun. 18, 2008, the disclosure of which thisapplication hereby incorporates by reference. U.S. patent applicationSer. No. 12/426,210 also relates to and claims the benefit of U.S.Provisional Patent Application Ser. No. 61/045,962, titled “Iris TextureQuality Enhancement from Video Sequences”, and filed on Apr. 17, 2008,the disclosure of which this application hereby incorporates byreference. U.S. patent application Ser. No. 12/426,210 also relates toand claims the benefit of U.S. Provisional Patent Application Ser. No.61/045,958, titled “Computationally Efficient Feature Extraction andMatching Iris Recognition”, and filed on Apr. 17, 2008, the disclosureof which this application hereby incorporates by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates, in general, to an image processing andbiometric recognition technique. In particular, the present invention isa system and method for representing an iris image to enhance therecognition technique.

2. Description of the Related Art

A biometric recognition system operates by acquiring biometric data froman individual, extracting a feature set from the acquired data, andcomparing this feature set against a template set in a database. Themost common biometric data that prior art biometric recognition systemsacquire include fingerprints, retinas, voice, and iris. Since the irishas the unique characteristic of very little variation over anindividual's life and a multitude of variation between individuals, irisdetection is one of the most accurate and secure means of biometricidentification. Furthermore, since iris-based recognition systems havebecome more user-friendly, iris detection is not only one of the leastinvasive detection methods, but also cost-effective.

The prior art describes capturing an image of an eye and analyzing theimage to produce an iris code. The prior art does not describe locatingthe iris in an image of an eye and representing the iris as aone-dimensional signal or iris string as disclosed herein. The prior artalso does not describe using the iris string for feature extraction,encoding, and matching.

Iris recognition to identify a subject, such as a human, animal, or thelike, has been proposed for more than 20 years and has been the subjectof numerous prior art publications. The prior art includes many detailedillustrations of the idea about how to set up equipment for irisacquisition. This is a factor in practical iris recognition systembecause the iris is a very small area to detect in comparison to a face.The subject's face can be captured very easily and non-intrusively, butif we want to capture an iris image, the task became not so trivial.Therefore, how to setup the cameras and the lighting in order to capturea high quality iris image is an important factor. The prior art alsodescribe the adoption of a digitalized controlled circuit where thecamera and lighting are all controlled by a central processing unit. Theprior art further proposes a new way of capturing an iris image so thatthere is no need to position the camera very close to the subjects,which greatly enhances the usability of iris recognition system in manypractical situations. For iris segmentation work, the prior art onlydescribes the use of a boundary detection algorithm or edge detectionalgorithm to localize the pupil. The prior art describes first findingthe limbic boundary and pupil boundary, and at last, localize the eyelidboundary. However, the prior art does not describe how to match twoirises and produce a score of likeness, and how much confidence can bebased on the chosen threshold. The prior art only provides exemplaryalgorithms that may perform well in iris recognition, for example, thewell-known Fisher LDA algorithm. The prior art also does not provide asubstantial statistical analysis about how good iris recognition canperform in turns of False Acceptance Rate (FAR) and False Reject Rate(FRR).

However, the prior art does not describe how to capture the iris imagein order to achieve such high recognition results. The prior art doesnot describe how good the iris image has to be in order to produce aclear iris code which can represent every detail in the iris pattern.Furthermore, the prior art describes methods to achieve such highquality iris images that involve the subject sitting on a chair, puttingtheir head on a wooden rack, and keeping their eyes wide open while thecamera takes the picture. This prior art method of iris acquisition willdrastically reduce the practicality of the iris recognition system.

The prior art demonstrates that high quality iris acquisition givesresults of good biometric matching, but reduces the practicality of thesystem. However, systems that can capture iris images in non-intrusivemanners usually reduce the iris image quality and inevitably downgradethe iris recognition performance. The iris image quality and the ease ofuse of the iris acquisition system are important factors for asuccessful biometric identification system; yet it seems that they aretwo factors usually system designers have to tune to trade them off. Thepresent invention of image enhancement technology acquires superhigh-resolution iris images while restricted conditions for the subjectsare minimized during iris acquisition stage. The present inventionachieves both of these goals (high quality iris image and thefriendliness of the system), while also achieving high performance ofiris recognition system.

SUMMARY OF THE INVENTION

Aspects of the present invention provide a method and system foruniquely identifying a subject based on an iris image. In oneembodiment, after obtaining the iris image, the method produces afiltered iris image by applying filters to the iris image to enhancediscriminative features of the iris image. The method analyzes anintensity value for pixels in the filtered iris image to produce an iriscode that uniquely identifies the subject.

In another embodiment, the method creates a segmented iris image bydetecting an inner and outer boundary for an iris region in the irisimage, and remapping pixels in the iris region, represented in aCartesian coordinate system, to pixels in the segmented iris image,represented in a log-polar coordinate system, by employing a logarithmrepresentation process.

In another embodiment, the method creates a one-dimensional iris stringfrom the iris image by unfolding the iris region by employing a spiralsampling method to obtain sample pixels in the iris region, wherein thesample pixels are the one-dimensional iris string.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram that illustrates one embodiment of an irisrecognition device that performs the present invention.

FIG. 2 is a flow chart that illustrates one embodiment of the basicapproach to a feature extraction process to generate an iris code forthe present invention.

FIG. 3 is a flow chart that illustrates one embodiment of a detailedimplementation of the feature extraction process that is acomputationally efficient feature extraction and encoding process forthe present invention.

FIG. 4 is a flow chart that illustrates one embodiment of an iris codematching process for the present invention.

FIG. 5 is a flow chart that illustrates one embodiment of the irissegmentation process for the present invention.

FIG. 6 illustrates the conversion of the coordinate system in an imagefrom the log-polar coordinate to Cartesian coordinate.

FIG. 7 illustrates a raw eye photo and various embodiments of irislogarithm representation of the present invention.

FIG. 8 is a flow chart that illustrates one embodiment of the processfor a typical iris recognition system of the present invention.

FIG. 9 is a flow chart that illustrates one embodiment of an iristexture quality enhancement process of the present invention.

FIG. 10 is an exemplary set of raw iris images and the correspondingiris texture patterns in log-polar coordinates of the present invention.

FIG. 11 is an exemplary illustration of the process of up-sampling animage.

FIG. 12 is an exemplary illustration of the process of breaking down abig image into patches.

FIG. 13 is an exemplary illustration of the process of matching andalignment of two patches.

FIG. 14 is an exemplary illustration of the process of pixel valueinterpolation from scene images to up-sampled template image.

DETAILED DESCRIPTION OF THE INVENTION

Computationally Efficient Feature Extraction and Matching IrisRecognition

The disclosed iris recognition process is a computationally efficientmethod for feature extraction and matching. The process generates aperson-specific code that attempts to identify a unique iris pattern inone eye from an iris pattern in another eye. A system that performs thisprocess will have the ability to identify a person from another personvery quickly and robustly using a computationally efficient process. Thedisclosed invention employs operations that are relativelycomputationally inexpensive and simple, such as differencing,thresholding, and simple filtering. The disclosed invention improvesupon the prior art conventional methods and processes iris data at ahigher throughput rate.

FIG. 1 is a block diagram that illustrates one embodiment of an irisrecognition device that performs the present invention. As shown in FIG.1, the iris recognition device 100 is a general-purpose computer. A bus102 is a communication medium that connects a central processor unit(CPU) 105, data storage device 110 (such as a disk drive, flash drive,flash memory, or the like), input device 115 (such as a keyboard,keypad, touchscreen, or the like), output device 120 (such as a monitor,graphic display, or the like), and memory 125.

The CPU 105 can be a commercially available or custom microprocessorthat performs the disclosed methods by executing the sequences ofoperational instructions that comprise each computer program residentin, or operative on, the memory 125. The reader should understand thatthe memory 125 may include operating system, administrative, anddatabase programs that support the programs disclosed in thisapplication. The memory 125 is representative of the overall hierarchyof memory devices containing the software and data used to implement thefunctionality of the computer 100. The memory 125 can include, but isnot limited to, cache, ROM, PROM, EPROM, EEPROM, flash memory, SRAM, andDRAM. In one embodiment, the configuration of the memory 125 of the irisrecognition device 100 includes a feature extraction program 130,matching program 135, iris segmentation program 140, and iris texturequality enhancement program 145. The feature extraction program 130performs the method of the present invention disclosed in detail in FIG.2 and FIG. 3. The matching program 135 performs the method of thepresent invention disclosed in detail in FIG. 4. The iris segmentationprogram 140 performs the method of the present invention disclosed indetail in FIG. 5. The iris texture quality enhancement program 145performs the method of the present invention disclosed in detail in FIG.8 and FIG. 9. These computer programs store intermediate results in thememory 125, or data storage device 110. In another embodiment, thememory 425 may swap these programs, or portions thereof, in and out ofthe memory 125 as needed, and thus may include fewer than all of theseprograms at any one time.

FIG. 2 is a flow chart that illustrates one embodiment of the basicapproach to a feature extraction process to generate an iris code forthe present invention. As shown in FIG. 2, the feature extractionprocess receives an iris image 210 as an input and produces an iris code240 as an output. The iris image 210 includes an original iris imagewithout any preprocessing, a segmented iris image, or the like.

The process shown in FIG. 2 begins by loading the iris image 210 into amemory of a general-purpose computing device, such as the irisrecognition device 100 shown in FIG. 1. The basic feature extractionprocess shown in FIG. 2 includes a filtering step 220 and an encodingstep 230. The filtering step 220 is the process of enhancing ortransforming the iris image 210 into filtered data, which is a suitablefeature space for the encoding step 230. The filtering step 220includes, but is not limited to, convolution, intensity normalization,or the like. The encoding step 230 is the process of performing a set ofoperations on the filtered data to construct the iris code 240. Theencoding step 230 includes, but is not limited to, applying threshold,performing various types of quantization, or the like.

FIG. 3 is a flow chart that illustrates one embodiment of a detailedimplementation of the feature extraction process that is acomputationally efficient feature extraction and encoding process forthe present invention. As shown in FIG. 3, the process receives an irisimage 310 as an input and produces an iris code 360 as an output. Theiris image 310 includes an original iris image without anypreprocessing, a segmented iris image, or the like.

The process shown in FIG. 3 begins by loading the iris image 310 into amemory of a general-purpose computing device, such as the irisrecognition device 100 shown in FIG. 1. If the iris image 310 is asegmented image, the image format is in a two-dimensional format sampledfrom the original iris employing a mapping function including, but notlimited to, log-polar segmentation, Cartesian segmentation, or the like.If the mapping function employs log-polar segmentation, a first axismaps to a region that is radially concentric to the pupil or iris, and asecond axis maps a linear distance away from the origin of the pupil oriris. If the mapping function employs Cartesian segmentation, the axisremains in the original domain; however, the iris region is remapped tofit within two concentric circles of fixed radii.

The process shown in FIG. 3 applies various filtering techniques 320 tothe segmented image to produce a filtered iris image. Applying thefilters serves to enhance certain discriminative features of the iris.The filters include, but are not limited to, a lowpass filter, bandpassfilter, highpass filter, edge enhancement, or the like.

The process shown in FIG. 3 continues by resizing 330 the filtered irisimage to yield a resized iris image. In one embodiment, the resizingstep 330 downsamples the filtered iris image by interpolating fromintensity values in the normalized iris image. The interpolation methodsinclude, but are not limited to, nearest neighbor, bilinear, bicubic, orthe like.

The process shown in FIG. 3 continues by processing the resized imagethrough a differencing operation 340 to generate a set of localdifference images to yield a local difference iris image. In oneembodiment, the process takes the difference between a pixel of theresized image and a specific neighbor to that pixel. For example, thedifference may be taken between the intensity of a pixel and itsimmediate adjacent pixel to the right, left, up, down, or diagonal. Inanother embodiment, the difference calculation is not limited todirectly adjacent neighbors and may extend further. For example, thedifference may also be taken between the intensity of a pixel and itsnon-adjacent neighbor. Furthermore, the resulting local difference irisimage may also be processed again using another local differencingprocess resulting in a set of local difference iris images.

The process shown in FIG. 3 continues by encoding 350 the set of localdifference iris images, and quantizing them to create the iris code 360.In one embodiment, the set of local difference iris images are a singlelocal difference iris image. The encoding step 350 employs variousmethods including, but not limited to, applying a threshold,quantization, a combination of these methods, or the like. In oneembodiment, the encoding step 350 applies a threshold such that if thedifference is greater than the threshold value, it is assigned a one (1)and if the difference is less than the threshold value, it is assigned azero (0). If that difference is equal to the threshold value, theconflict can be resolved in several ways such as assigning itarbitrarily but consistently to a one (1) or a zero (0). In anotherembodiment, the encoding step 350 employs quantization based on whethera difference value is greater than or less than a threshold value. Inyet another embodiment, the encoding step 350 employs quantization basedon whether the magnitude of the difference is greater than or less thana threshold value.

In one exemplary embodiment, the computationally efficient featureextraction and encoding process shown in FIG. 3 comprises the followingsteps. The process obtains a segmented iris image 310 using a log-polarmethod. The process applies a filter 320 to the segmented iris imageusing a two-dimensional Gaussian filter with different sigma values forthe Gaussian in the X and Y directions to yield a filtered iris image.The process resizes 330 the filtered iris image to a dimension of 25pixels by 180 pixels using a “Nearest Neighbor” interpolation method toyield the resized iris image. The process computes the local difference340 of each pixel in the resized iris image with its adjacent neighborto the right to yield a local difference iris image. For the pixelslocated in the last column of the resized iris image, the processcircularly accesses the pixels on the first column to take thedifference. The process computes the local difference again using thesame process, but this time from the local difference iris image. Thiscomputation will yield the second local difference iris image. The twolocal difference iris images comprise the set of local difference irisimages. The process creates the iris code 360 by concatenating the setof local difference iris images and applying a threshold on each pixel350. If a pixel intensity of the concatenated local difference irisimages is above zero (0), the pixel value is set to one (1). Likewise,if a pixel intensity is less than or equal to zero (0), the pixel valueis set to zero (0). Lastly, the process converts the thresholded imageinto a binary bitmap of ones (1) and zeros (0) into a format where thepixel location and the thresholded value at that pixel location arepreserved. This exemplary embodiment is provided to further describe andclarify the present invention. In other exemplary embodiments, theprocess may apply different filters, resizing operations, difference andquantization, and encoding methods. Furthermore, the process may modifythe order of the operations or add additional operations in-between thesteps described in the process.

FIG. 4 is a flow chart that illustrates one embodiment of an iris codematching process for the present invention. As shown in FIG. 2, the iriscode matching process describes the approach for computing a matchingscore 460 between a first iris code 410 and first iris mask 412 pair anda second iris code 414 and second iris mask 416 pair. The first irismask 412 and second iris mask 416 are created by a prior art process. Aniris mask is a bitmap that has the same dimensions as the iris code. Itdesignates the indices of the iris code where it is considered valid.Regions of the iris code might be considered invalid for cases such aseyelid occlusion, eyelash occlusion, image artifacts on regions of theiris code, or the like.

The process shown in FIG. 4 begins by loading the first iris code 410and first iris mask 412 pair and a second iris code 414 and second irismask 416 pair into a memory of a general-purpose computing device, suchas the iris recognition device 100 shown in FIG. 1. An iris maskrepresents the valid regions (i.e., list of indices) of the iris code touse for matching. For example, if the iris mask at index i is true, thenuse index i of the iris code for matching. Likewise, if the value ofiris mask at index i is false, then neglect index i of iris code formatching. The method then combines 420 the first iris mask 412 andsecond iris mask 416 such that the resulting combined iris maskspecifies the indices of the iris masks where both iris codes are valid.The method counts the total number of valid indices 430 in the combinedmask. This value will be referred to as the number of valid bits. Whereboth iris codes are valid based on the combined iris mask, the methodcounts the number of indices where the two iris codes differ 440. Thisvalue will be known as the number of mismatched bits. Where the firstiris code 410 and the second iris code 414 match for the indices denotedas valid in the combined iris mask, the method counts the number of bitsthat match. This value will be known as the number of matched bits. Inone embodiment, the process then computes 450 the matching score 460 bydividing the number of matched bits by the number of valid bits. Inanother embodiment, the process then computes 450 the matching score 460by dividing the number of mismatched bits by the number of valid bits.

Iris Logarithm Representation

FIG. 5 is a flow chart that illustrates one embodiment of the irissegmentation process for the present invention. As shown in FIG. 5, theiris segmentation process includes two steps, iris boundary fitting, andlogarithm representation. The iris segmentation process is a two-stepprocess, detecting two approximate circles around a pupil and a sclera,and refining the approximate circles to detect pupil and scleraboundaries that are more accurate.

The process shown in FIG. 5 begins by loading the iris image 510 into amemory of a general-purpose computing device, such as the irisrecognition device 100 shown in FIG. 1. The iris segmentation processthen employs circle detection 520, or other methods, to detect twoapproximate circles that define the inner (pupil) boundary and the outer(sclera) boundary of the iris. FIG. 5 includes an exemplary illustrationof employing circle detection 525. Occlusion by the eyelids andeyelashes often make it difficult to obtain a whole iris region. Inparticular, the upper eyelids and eyelashes often partly or severelyocclude the upper outer boundaries of the iris region. To get successfulresults of iris recognition in spite of these occlusion problems, thepresent invention detects and excludes the occlusion regions duringsegmentation. After detecting two approximate circles around a pupil anda sclera, the iris segmentation process refines the approximate circlesby selecting a number of initial points on the circles 530, andemploying iris boundary fitting 540. FIG. 5 includes an exemplaryillustration of selecting initial points on the circles 535, andemploying the iris boundary fitting 545. The present invention usesenergy-minimizing methods or splines, such as active contours, snakes,or other morphological, and image and computer vision processingmethods, to detect boundaries that are more accurate. The true irisregion has an annular shape, resembling a ring. In various embodiments,the annular shape may be circular, elliptical, or arbitrary shaped. Inanother embodiment, the shape of the approximate region for the pupildiffers from the shape of the approximate region for the sclera. Afterdetecting the true iris region, the iris segmentation process thenemploys a logarithm representation process 550 to convert the true irisregion to a rectangular shape. FIG. 5 includes an exemplary illustrationof the true iris region converted to a rectangular shape 555.

The annular shape of the extracted iris region allows the logarithmrepresentation process of the present invention to unfold the irisregion by remapping the coordinate system in an image from the Cartesiancoordinate system to the log-polar coordinate system as shown in FIG. 6.FIG. 6 depicts an original image 610 of the detected true iris image andan output image 620 converted to a rectangular shape. To decide theintensity value at each pixel (h, α) in the output rectangular image620, the corresponding pixel (x, y) in an original image 610 iscalculated using r and α; α is easily obtained by the horizontal indexin the output image, and r is calculated by r=s exp(βh), where s is ascaling factor to map the vertical indices in an output image linearlyinto [0, r_(out)−r_(in)]. In another embodiment, r is calculated by r=slog_(b)(βh), where b is the base. Finally, (x, y) corresponding to (h,α) is calculated by x=(r+r_(in)) cos α+x₀ and y=(r+r_(in)) sin α+y₀. Ifx and y are integers, the pixel (h, α) can easily take the sameintensity value as in (x, y). Actually, x and y are usually notintegers, so we take the weighted sum of the intensities in the fourpixels around the exact (x, y) by linear interpolation.

FIG. 7 illustrates a raw eye photo and various embodiments of irislogarithm representation of the present invention. FIG. 7 shows anoriginal eye image 710 with a log-polar coordinate overlay that includesconcentric circles 711, 712, 713, 714, 715, where the innermostconcentric circle 711 is at the pupil boundary, and the outermostconcentric circle 715 is at the sclera boundary. FIG. 7 also illustratesa first embodiment 720 and a second embodiment 730 of the iris logarithmrepresentations of the present invention. Various embodiments of theiris logarithm representations change the parameters of the equationsassociated with the coordinate system shown in FIG. 6 to obtaindifferent logarithm sampling. As shown in FIG. 7, the first embodiment720 and the second embodiment 730 change the β parameter. By varying theparameters in the equations associated with the coordinate system shownin FIG. 6, the present invention may tune the log sampling to emphasizemore of the iris region near the pupil region boundary, or more of theiris region near the sclera region boundary. As shown in FIG. 7, theconcentric circles 711, 712, 713, 714, 715 of the log-polar coordinateoverlay for the original eye image 710 appear as straight lines in thefirst embodiment 720 and second embodiment 730 of the iris logarithmrepresentations. The straight lines 721, 722, 723, 724, 725 of the firstembodiment 720 correspond, respectively, to concentric circles 711, 712,713, 714, 715 for the original eye image 710. Similarly, the straightlines 731, 732, 733, 734, 735 of the second embodiment 730 correspond,respectively, to concentric circles 711, 712, 713, 714, 715 for theoriginal eye image 710. These straight lines illustrate that the secondembodiment 730, when compared to the first embodiment 720, shows moreemphasis and zooming of the inner annular portion (between the innermostconcentric circle 711 and the next outer concentric circle 712) of theiris region (i.e., the portion of the iris region near the pupilboundary) and less emphasis and zooming toward the outer annular portion(between the outermost concentric circle 715 and the next innerconcentric circle 714) of the iris region (i.e., the portion of the irisregion near the sclera boundary). The change in emphasis and zooming hasa number of advantages. First, since the portion of the iris region nearthe pupil boundary is thought to be more discriminative for certainpopulations, the emphasis and zooming of the present invention enhancesthe ability of the iris logarithm representation as a biometricrecognition technique. Second, the emphasis and zooming of the presentinvention makes the iris logarithm representation less prone to eyelashand eyelid occlusion. Furthermore, since the emphasis and zooming of thepresent invention is variable by tuning the parameters of the logsampling, the present invention analyzes each eye image, independently,and tunes the parameters to select the log sampling that will producethe best iris logarithm representation for that eye image.

Segmentation of Iris Region for Iris Recognition Using Spiral Unwrapping

In another embodiment, the present invention represents the located irisas a one-dimensional signal or iris string by taking samples from theiris image while spiraling outward from the pupillary boundary to thesclera boundary. After obtaining the iris string representation, featureextraction and iris coding processes can take place usingone-dimensional filters or other one-dimensional feature extractionmethods, resulting in a very computationally efficient featureextraction and matching process. The computational efficiencyimprovements will allow processing of iris image data at greaterthroughput rates than with prior art methods.

Referring again to FIG. 5, after circle detection 520, and boundaryfitting 530, 540 to detect the true iris region, in one embodiment, theiris segmentation process takes samples of the iris image spiralingoutward from the pupil boundary. The present invention uses thediscovered inner circular boundary (pupil boundary) and the outercircular boundary (sclera boundary) of the true iris region, to convertthis region to a one-dimensional representation. In various embodiments,the inner boundary is elliptical or arbitrary shaped, and the outerboundary is elliptical or arbitrary shaped. Thus, after detecting thetrue iris region, an extracted iris region has an annular shape,resembling a ring, and may be circular, elliptical, or arbitrary shaped.The annular shape enables the present invention to unfold the irisregion by sampling points. In one embodiment, the spiral sampling isoutward from the pupillary boundary toward the sclera boundary. Inanother embodiment, the spiral sampling is inward from the scleraboundary toward the pupillary boundary. Since the pupil and iris may notbe concentric, the present invention normalizes the spiral samplingmethod to maintain a consistent number of samples between the pupilboundary and the iris boundary at every angle θ. Thus, one property ofthe one-dimensional iris string representation is that it is always ofthe same length regardless of the size of the iris or pupil, andregardless of the degree of pupil dilation. The iris stringrepresentation may also have a degree of rotational invariance infurther processing on this domain by a simple process of shifting theone-dimensional iris string to the left or right by fixing the anglestep between adjacent samples. In various embodiments, the spiralsampling method may employ a linear, Archimedean, or logarithmic spiral.In another embodiment, since the pupil and the iris may not beconcentric, the spiral sampling method chooses a spiral shape thatobtains a selected number of turns on each side. The spiral unwrappingcan be an increasing function of radius as angle increases, but can alsobe a complete dense sampling which samples as a function of completeangular sampling and then increasing the radius for another completeangular sampling, so that the spiral increasing radius can be a functionof a complete angular slice sampling of iris.

Iris Texture Quality Enhancement from Video Sequences

The present invention records the iris of subjects with a video capturedevice. The video capture device captures the irises sequentially.Therefore, many images of the same iris have been taken in differenttime stamps, with those time stamps spaced with short intervals.

The present invention takes a series of iris video images, findsinternal information among each image, fuses the internal information inan intelligent way, and generates a new iris images that have much moredetailed information about iris patterns than any of the input irisimages.

The present invention analyzes the information content of the inputimages by decomposing the input iris images into some form ofrepresentations with two dimensional signal basis. The decompositionprocedure in the algorithm may include, but is not limited to, Fourieranalysis (two-dimensional Fourier Transform), wavelet decomposition,principal component analysis, Fisher's linear discriminant analysis,fractal geometry, and the like.

The information fusion process of the present invention may employ manyalternative algorithms, all of which all can serve to achieve the samegoal. The choices of algorithm include, but is not limited to, linearcombination of pixel-wise information, Artificial Neural Network (ANN),Hidden Markov Model (HMM), Probabilistic Graphical Model, and the like.

The video capture device of the present invention includes, but is notlimited to, digital cameras of any brand, digital camcorders of anybrand, traditional film cameras that have the ability to convert theimages to a digital form, and the like.

The digital form of the present invention include digital images thatcontains pixel-wise information of iris pattern and includes, but is notlimited to, the popular digital image format used in modern computersystems, for example, with the file extensions like bmp (Bitmappedimage), eps (encapsulated postscript), tiff or tif (Tagged Image FileFormat), jpeg or jpg (Join PhotoGraphic Experts Group), gif (GraphicsInterchange Format), pict, and the like.

The iris acquisition procedure of the present invention may include asubject standing or sitting at a fixed place, may include the subjectplacing their chin against a fixed apparatus, and may include thesubject looking at a pre-defined location or point. However, the methodof the present invention minimizes the restrictions on the irisacquisition procedure.

FIG. 8 is a flow chart that illustrates one embodiment of the processfor a typical iris recognition system of the present invention. The maincomponent or function modules are depicted as rectangles, with theexemplar images by its right side. Most parts of the iris recognitionframework mentioned here is following the framework described in priorart patents, though the matching and classification can be replaced byother modern machine learning algorithms.

The process shown in FIG. 8 begins by loading a raw iris image 810 intoa memory of a general-purpose device, such as the iris recognitiondevice 100 shown in FIG. 1. An exemplar raw iris image 815 is shown inFIG. 8. The raw iris image 810 is fed into the iris localizer module820, which finds the inner boundary (between iris and pupil) and outerboundary (between iris and sclera) of an iris. An exemplar eye image 825after processing by the iris localization module 820 is shown in FIG. 8.

The results of iris localization will tell us the location of everypoint on inner and outer boundaries. After this information is given,the iris pattern can be “unwrapped” from an annular shape (in theoriginal Cartesian coordinate) to a rectangular shape in the irispattern unwrapping module 830. An exemplar unwrapped iris pattern 835 isshown in FIG. 8. The new rectangular shape is arranged in log-polarcoordinate. The horizontal axis denotes the angle θ between thehorizontal line (which is of zero degree) and the line connecting centerof pupil to the current pixel. Therefore, the value of horizontal axisspans from 0 to 360 degrees. The vertical axis denotes the distance rbetween current pixel and the closest pixel on the pupil boundary. Theiris pattern unwrapping module 830 (i.e., coordinate transformationprocess) uses the logarithm representation 550 shown in FIGS. 5, 6, and7, or a prior art transformation process.

The advantage of coordinate transformation is to provide a normalizediris pattern which is independent of the influence of pupil dilation orcontraction, and also independent of the problem caused bynon-concentricity of the pupil and iris. One can understand thisadvantage by a simple imagination. Assume the pupil of the exemplar rawiris image 815 dilates to double size, if we compare the two irispattern in Cartesian coordinate (before unwrapping), the differencesbetween them will be huge because some part of iris pattern inun-dilated version is mapped to the pupil area in dilated version. But,if the iris pattern is transformed to log-polar coordinate, the two irispattern will maintain great similarity.

Another advantage of coordinate transformation is that it makes easierto solve the problem when the iris image (or the raw eye image 815) isslightly rotated. As one can imagine, the head position of subjectscannot always be exactly the same position when the picture is taken.Therefore, it is highly possible that the raw eye image may have arotational shift compared to image stored in iris database. Again, bysimply comparing image in Cartesian coordinate cannot solve the problembecause the distance between these two patterns will be large. But, ifwe transform iris pattern to log-polar coordinate, the rotational shiftwill become translation in x coordinate. Hence, one can simply shift thereference image (or the test image) a certain amount of pixel beforematching the two patterns, and choose the best result among the multiplecomparisons.

After the iris pattern is transformed into log-polar coordinate, it goesinto the feature extraction module 840 to extract features from theunwrapped image 835. In this stage, different kinds of bandpass filterscan be applied to the unwrapped iris image 835, and the output will bestored as features of the iris pattern. Note that different prior artuses different type of filters. For the purpose of this invention, how afeature is extracted is not the main concern. Therefore, any kind offeature extraction can be utilized in this iris recognition framework.An exemplar iris feature image 845 after feature extraction is shown inFIG. 8.

After the iris feature image 845 is extracted, the iris recognitionframework can register this iris feature image 845 with an iris database850. In one embodiment, when the iris recognition framework performsidentification or verification of a subject, the iris feature image 845would need to be retrieved to the system to allow classification module860 to start its job. According to the type of feature, differentclassification schemes may be utilized. For the purpose of thisinvention, how classification is performed is not the main concerneither. Any classification algorithm will do as long as it works for thecorresponding feature scheme.

After the classification, an identification/verification result 870 isdetermined based on the lowest distance or highest similarity score. Anexemplar decision about the subject's identity would identify the nameof the person associated with the raw iris image 815.

FIG. 9 is a flow chart that illustrates one embodiment of an iristexture quality enhancement process of the present invention. As shownin FIG. 9, an iris acquisition device captures a series of picturestaken from the same subject. The iris texture quality enhancement module950 works as a middle layer between the iris pattern unwrapping module830 and the feature extraction module 840 shown in FIG. 8. Theassumption is that we have input raw eye image from a video sequence.Therefore, the number of the same iris patterns should be more than atleast three. Let us say there are four iris patterns of the same subject(and the same eye) recorded in this video sequence. After irislocalization 820 and processing by the iris pattern unwrapping module830, there should be four rectangular images 910, 920, 930, 940 similarthe exemplar unwrapped image 835. The iris texture quality enhancementmodule will take these four images 910, 920, 930, 940 as input andproduce one high quality iris image 960, also in rectangular form (polarcoordinate) which has the same resolution as the input images 910, 920,930, 940. After that, the newly generated high quality iris image 960will be used in feature extraction and classification framework.

FIG. 10 is an exemplary set of raw iris images and the correspondingiris texture patterns in log-polar coordinates of the present invention.As shown in FIG. 10, the first raw iris image 1010 and the second rawiris image 1030 were taken of the same subject, with a video recordingdevice. As one can imagine, in the whole video sequence of eye images,some of them might be taken within an effective range of focus zone, andhave higher quality, but some may not. The first raw iris image 1010 isan example of a high quality picture. On the contrary, the second rawiris image 1030 is an example of a low quality image. After performingiris localization 820 on the first raw iris image 1010, the irisboundary can be found, and the annular iris region can be unwrapped to arectangular map as shown as the first iris texture pattern in log-polarcoordinates 1020. The same procedure performed on the second raw irisimage 1030 results in another rectangular map as shown as the secondiris texture pattern in log-polar coordinates 1040. By comparing thefirst iris texture pattern in log-polar coordinates 1020 with the secondiris texture pattern in log-polar coordinates 1040, it is very clearthat the first iris texture pattern in log-polar coordinates 1020 hasmuch more detailed information about iris pattern (for examples, thefurrow and ridges in the iris pattern) than the second iris texturepattern in log-polar coordinates 1040. Experimental result also tellsthe same results. The third raw iris image 1050 shown in FIG. 10 isanother raw eye image of the same subject, taken with another highresolution camera. In the third iris texture pattern in log-polarcoordinates 1060, one can see much more detailed information than ineither the first iris texture pattern in log-polar coordinates or thesecond iris texture pattern in log-polar coordinates. After performing amatching algorithm between the first iris texture pattern in log-polarcoordinates 1020 and the third iris texture pattern in log-polarcoordinates 1060, and between the second iris texture pattern inlog-polar coordinates 1040 and the third iris texture pattern inlog-polar coordinates 1060, similarity scores between these two pairsare obtained. The similarity score between the first iris texturepattern in log-polar coordinates 1020 and the third iris texture patternin log-polar coordinates 1060 is 0.5288. The similarity score betweenthe second iris texture pattern in log-polar coordinates 1040 and thethird iris texture pattern in log-polar coordinates 1060 is 0.3825.Thus, the first iris texture pattern in log-polar coordinates 1020 andthe third iris texture pattern in log-polar coordinates 1060 are moresimilar than the second iris texture pattern in log-polar coordinates1040 and the third iris texture pattern in log-polar coordinates 1060.

From this experiment, one can see that the similarity score may go downa great level for the same subject, with exactly the same irisacquisition device, just because of the clarity of the former image orthe blurring effect of the latter image. There might be many reasonswhich account for blurring of the images. For example, images areblurred when the positioning of the subject is out of the focus zone,when either the subject or camera is moving, or when there is stronglevel of noise. All of these difficulties may arise during practicaliris recognition deployment. Therefore, it is very important to enhancethe quality of iris images before it goes into feature extraction andmatching stages.

FIG. 11 is an exemplary illustration of the process of up-sampling animage. The first step in the image quality enhancement algorithm is tochoose one image among all input images to be the template image. Afterthe template image has been chosen, up-sample it into a double-sizedimage, with zeros (0's) interpolated in between every pair of the pixelsin template image. Since every image in its digital form can berepresented as a two dimensional real value matrix, the illustrationshown in FIG. 11 starts with a very simple 2×2 matrix 1110. Afterup-sampling process, the up-sampled 2×2 matrix 1110 becomes a 4×4 matrix1120. One can see that the 4×4 matrix 1120 inserts zeros (0's) inbetween every pair of pixels in the 2×2 matrix 1110. Note that theup-sampling process will double the size of the image.

A more realistic up-sampling example is shown in the iris templateimages shown in FIG. 11. The first iris template image 1130 is a choseniris template image. After the up-sampling process, the first iristemplate image 1130 becomes a double-sized second iris template image1140, with zeros (0's) inserted in between every pair of pixels of thefirst iris template image 1140.

FIG. 12 is an exemplary illustration of the process of breaking down abig image into patches. All other input images that are not chosen to betemplate are called scene images. The second step of image qualityenhancement procedure is to break all input images (including templateand scene image) into pictures of smaller sizes. These smaller sizeimages are called “patches” in the domain of image processing. As shownin FIG. 12, the first image 1210 is one of the input images with a sizeof 60×360. If we break the first image 1210 into patches of smallersizes, say 30×15, the second image 1220 will be the output image, whichincludes a total of 48 patches (48=2×24). Note that the size of both theiris image and the patches are just an example. The algorithm covered inthis patent is not restricted in the size of the image and patches. Thatmeans, any image size and patch size will fit in the algorithm of thepresent invention.

FIG. 13 is an exemplary illustration of the process of matching andalignment of two patches. The third step for image quality enhancementis to match the patterns in one patch to the patch from another image.As shown in FIG. 7, the first input image 1310 and the second inputimage 1320 are two patches from different images. The first input image1310 is one patch from a template image, and the second input image 1320is another patch which is in a corresponding location of the first inputimage 1310, from scene image. Because of the variability of the irispatterns, template and scene images may not match and align to eachother. What makes the problem more difficult is that these two imagesmay have local distortion scattered all over the entire image plane. Inother words, if we want to align template and scene image, the alignmentprocess has to be done locally, instead of globally. It justifies why wewould like to break the iris pattern into many smaller sized image(patches) because it helps us to match and align pattern locally.

By performing normalized correlation, one can measure at which locationtwo images match each other best. As shown in FIG. 13, the region in thefirst input image 1310 marked by a rectangle 1315 clearly matches to theregion in the second input image 1320 marked by a rectangle 1325. Oneimportant thing during matching is that the goal of matching is to matchthe detailed variation in texture of the two images, not the absolutepixel value. In the case of the example images shown in FIG. 13, whenlooking closely, one can see that the intensity of the two rectangleregion in the patch in the first input image 1310 and the second inputimage 1320 are not exactly the same. The intensity variation (contrast)in the rectangle region 1315 of the first input image 1310 seemssmoother than the intensity variation in the rectangle region 1325 ofthe second input image 1320. However, the texture patterns in those tworectangles are the same. As long as two texture patterns are the same,though absolute pixel value may be different, they can be matched toeach other.

Therefore, the first input image 1310 patch can be aligned with secondinput image 1320 patch to give the aligned image 1330. The aligned image1330 is a new pattern compared to either the first input image 1310 orthe second input image 1320. For the first input image 1310, the alignedimage 1330 provides new information outside of its southeast boundary.For the second input image 1320, the aligned image 1330 provides newinformation outside of its northwest boundary.

FIG. 14 is an exemplary illustration of the process of pixel valueinterpolation from scene images to up-sampled template image. Afterlocal patches from different images are matched and aligned, newinformation can be fused into the template image. This procedure isillustrated in FIG. 14. The 2×2 matrices 1410, 1420, 1430, 1440 shown inFIG. 14 represent the four input images to the iris pattern qualityenhancement module 950 shown in FIG. 9. The central 2×2 matrix 1410represents the up-sampled template image. The matching and alignmentprocess shown in FIG. 13 is performed as described above. Now, there arethree patches coming from three different scene images, all aligned withthe same template patch. Locating the position where central 2×2 matrix1410 is in the template patch, and retrieving the matrix values atcorresponding position of each aligned patch, one will get threedifferent matrices 1420, 1430, 1440. Because these three matrices 1420,1430, 1440 are from patches that already aligned with template patch,one can assert that they contain information in the corresponding regionin template image, which, in this case, is the central 2×2 matrix 1410.Therefore, if one would like to recover unknown information inup-sampled template patch 1410, one could fill the area of unknown (inthis case, the places that are occupied by zeros (0's)) with the valuesfrom corresponding locations in scene patches.

The process of information filling from the three scene matrix to thetemplate matrix can be done in many different ways. For example, directpixel value interpolation is one way. This invention is not bound by anyspecific way of information fusion. That means, all kinds of possibilityof information fusion is an option covered in this patent, as long as itcreates a new image based on combination of information derived fromscene and template images. In FIG. 14, the up-sampled template image1410 and the scene images 1420, 1430, 1440 are input to the informationfusion scheme 1450 process to produce the a resultant matrix 1460. Byrepeatedly doing the same thing for every location in up-sampledtemplate image, the holes (places that have zero value) in up-sampledtemplate will be filled up, and a new image with high quality isgenerated.

The last step for image quality enhancement is to down-sample theup-sampled template image. In this step, many different down-samplealgorithms can be used, for examples, nearest-neighbor algorithm,bilinear interpolation or bi-cubic interpolation. After this step, thenew image is down-sampled to the same size as input images, and alsocontains new information from all of the input images.

The invention can be used in all of the existed iris recognitionsystems, independent with what frame work the iris recognition systemadopts, what segmentation algorithm the system is using, and whatmatching algorithm the system is using. As long as the iris recognitionsystem can acquire more than one iris image, the invention can help toimprove the quality of the input iris image, and therefore, improve theperformance of iris recognition system.

The present invention is an algorithmic procedure that comprises takinginput as a series of iris video images, finding internal informationamong each image, fusing the internal information in an intelligent way,and generating a new iris image that have much more detailed informationabout iris patterns than any of the input iris images. The format ofinput images includes all digital images format used in modern computersystems. The format can be, but not restricted to, BMP (Bitmappedimage), EPS (encapsulated postscript), TIFF or TIF (Tagged Image FileFormat), JPEG or JPG (Join PhotoGraphic Experts Group), GIF (GraphicsInterchange Format), RLE (Run-Length Encoding), MSP (Microsoft Paint),PCX, PCD (Photo CD) and pict. The number of the input images, can be anynumber greater than one. The algorithmic procedure comprises picking uptemplate image, up-sampling the template image, breaking all inputimages into patches, matching and aligning patches between template andscene images, replacing the zeros in up-sample template image by fusinginformation from all the scene patches at corresponding location, anddown-sampling the up-sampled image. The method for selecting of thetemplate image, can have a number of variety. These variety includes,but not limited to, choosing randomly, choosing the template imageaccording to the time stamps of input images, choosing the templateimage according to the position of the subject in the capture volume,performing frequency domain analysis, then choose the image that has themost abundant high frequency components, and applying multi-scale,multi-orientation two dimensional band-pass filters to extract featuresfor all input images, then applying classifiers to separate the templateimage from scene images. The method for frequency domain analysis, caninclude, but not limited to two dimensional Fourier Analysis, threedimensional Analysis (2D image in addition to the sequential relationbetween images), and wavelet decomposition. The method for featureextraction, can include, but not limited to, using Gabor filters assignal basis to perform feature extraction, using 2D sine and cosinewaves as signal basis to perform feature extraction, principal ComponentAnalysis (PCA), two dimensional PCA, Fisher's Linear DiscriminantAnalysis (FLDA), two dimensional FLDA, independent Component Analysis(ICA), Non-Negative Matrix Factorization (NMF), and fractal geometry.The method for classification, can include, but not limited to, NearestNeighbors (NN), Artificial Neural Network (ANN), Support Vector Machine(SVM), Decision Tree, Correlation Filter, and first perform sub-spaceprojection, then followed by a classifier. The method for sub-spaceprojection, can include, but not limited to using Gabor filters assignal basis to perform feature extraction, using 2D sine and cosinewaves as signal basis to perform feature extraction, principal ComponentAnalysis (PCA), two dimensional PCA, Fisher's Linear DiscriminantAnalysis (FLDA), two dimensional FLDA, independent Component Analysis(ICA), Non-Negative Matrix Factorization (NMF), and fractal geometry.The method for classifiers can include, but not limited to, NearestNeighbors (NN), Artificial Neural Network (ANN), Support Vector Machine(SVM), Decision Tree, Correlation Filter, and first perform sub-spaceprojection, then followed by a classifier. The method for matching andaligning patches, can include but not limited to, two dimensionalcorrelation, two dimensional normalized correlation, correlationfilters, any of the previous methods applied on phase-only informationof the patches, and any of the previous methods applied onmagnitude-only information of the patches. The method for informationfusing from multiple scene images, can include but not limited to,replacing the old pixel values with pixel values from any scene image,letting new pixel value to be a linear combination of the pixel valuesfrom all scene images, and using non-linear optimization technique totune the parameters for nonlinear combination of the pixel values fromall scene images. The method for non-linear optimization, can includebut not limited to, conjugate gradient method, simulated annealing,Nelder-Mead method, genetic algorithm, artificial neural network, HiddenMarkov Model, and Probabilistic Graphical Model. The size of the imagesand the size of the patches, can be any positive integer number greaterthan 1. The apparatus for capturing iris images, can include but notlimited to, digital cameras of any brand, digital camcorders of anybrand, and traditional film cameras which has the ability to convert theimages to a digital form. The procedure for iris acquisition, may or maynot include the subject standing or sitting at a fixed place, may or maynot ask subjects to place their chin against a fixed apparatus, and mayor may not include the subject looking at a pre-defined location orpoint.

Although the disclosed exemplary embodiments describe a fullyfunctioning method and system for uniquely identifying a subject basedon an iris image of an eye of the subject, the reader should understandthat other equivalent exemplary embodiments exist. Since numerousmodifications and variations will occur to those reviewing thisdisclosure, the method and system for uniquely identifying a subjectbased on an iris image of an eye of the subject is not limited to theexact construction and operation illustrated and disclosed. Accordingly,this disclosure intends all suitable modifications and equivalents tofall within the scope of the claims.

1-22. (canceled)
 23. A method for creating a one-dimensional iris stringfrom an image of an eye of a subject, comprising: obtaining the image ofthe eye; detecting an iris region in the image of the eye, the irisregion having an inner boundary and an outer boundary; and unfolding theiris region by employing a spiral sampling method to obtain samplepixels in the iris region, wherein the sample pixels are theone-dimensional iris string.
 24. The method of claim 23, wherein thedetecting of the iris region further comprises: detecting a firstapproximate circle to define the inner boundary of the iris region,wherein the inner boundary is between a pupil of the eye and the iris;detecting a second approximate circle to define the outer boundary ofthe iris region, wherein the outer boundary is between the iris and asclera of the eye; applying an iris boundary fitting process to a numberof initial points on the first approximate circle to refine the innerboundary of the iris region; and applying the iris boundary fittingprocess to a number of initial points on the second approximate circleto refine the outer boundary of the iris region.
 25. The method of claim23, wherein the spiral sampling is outward from the inner boundarytoward the outer boundary.
 26. The method of claim 23, wherein thespiral sampling is inward from the outer boundary toward the innerboundary.
 27. The method of claim 23, wherein the unfolding of the irisregion further comprises: normalizing the spiral sampling method toobtain a consistent number of sample pixels between the inner boundaryand the outer boundary at every angle θ.
 28. A system for using acomputer to create a one-dimensional iris string from an image of an eyeof a subject, comprising: a memory device resident in the computer; anda processor disposed in communication with the memory device, theprocessor configured to: obtain the image of the eye; detect an irisregion in the image of the eye, the iris region having an inner boundaryand an outer boundary; and unfold the iris region by employing a spiralsampling method to obtain sample pixels in the iris region, wherein thesample pixels are the one-dimensional iris string.
 29. The system ofclaim 28, wherein to detect the iris region, the processor is furtherconfigured to: detect a first approximate circle to define the innerboundary of the iris region, wherein the inner boundary is between apupil of the eye and the iris; detect a second approximate circle todefine the outer boundary of the iris region, wherein the outer boundaryis between the iris and a sclera of the eye; apply an iris boundaryfitting process to a number of initial points on the first approximatecircle to refine the inner boundary of the iris region; and apply theiris boundary fitting process to a number of initial points on thesecond approximate circle to refine the outer boundary of the irisregion.
 30. The system of claim 28, wherein the spiral sampling isoutward from the inner boundary toward the outer boundary.
 31. Thesystem of claim 28, wherein the spiral sampling is outward from theinner boundary toward the outer boundary.
 32. The system of claim 28,wherein to unfold the iris region, the processor is further configuredto: normalize the spiral sampling method to obtain a consistent numberof sample pixels between the inner boundary and the outer boundary atevery angle θ.